Random Walk Distributed Dual Averaging Method For Decentralized Consensus Optimization
نویسندگان
چکیده
In this paper, we address the problem of distributed learning over a large number of distributed sensors or geographically separated data centers, which suffer from sampling biases across nodes. We propose an algorithm called random walk distributed dual averaging (RW-DDA) method that only requires local updates and is fully distributed. Our RW-DDA method is robust to the change in network topology and amenable to asynchronous implementation. The theoretical analysis shows the algorithm has O(1/ √ t) convergence for non-smooth convex problems. Experimental results show that our algorithm outperforms competing methods in real-world scenarios, i.e. when trained over non-iid data and in the presence of communication link failures.
منابع مشابه
Random Walk Distributed Dual Averaging Method For Decentralized Consensus Optimization
In this paper, we address the problem of distributed learning over a decentralized network, arising from scenarios including distributed sensors or geographically separated data centers. We propose a fully distributed algorithm called random walk distributed dual averaging (RW-DDA) that only requires local updates. Our RW-DDA method, improves the existing distributed dual averaging (DDA) method...
متن کاملDistributed Output Consensus via Lmi-based Model Predictive Control and Dual Decomposition
This paper proposes a distributed model predictive control method for solving an optimal consensus problem, where a system consists of networked multiple agents and the outputs of all agents converge to a common point. The problem is formulated as a convex optimization problem involving linear matrix inequalities, and then solved by using dual decomposition. In the proposed scheme, the state fe...
متن کاملApproximate Projection Methods for Decentralized Optimization with Functional Constraints
We consider distributed convex optimization problems that involve a separable objective function and nontrivial functional constraints, such as Linear Matrix Inequalities (LMIs). We propose a decentralized and computationally inexpensive algorithm which is based on the concept of approximate projections. Our algorithm is one of the consensus based methods in that, at every iteration, each agent...
متن کاملGossip Dual Averaging for Decentralized Optimization of Pairwise Functions
In decentralized networks (of sensors, connected objects, etc.), there is an important need for efficient algorithms to optimize a global cost function, for instance to learn a global model from the local data collected by each computing unit. In this paper, we address the problem of decentralized minimization of pairwise functions of the data points, where these points are distributed over the...
متن کاملThe Structure of Optimal and Near Optimal Target Sets in Consensus Models
We consider the problem of identifying a subset of nodes in a network that will enable the fastest spread of information in a decentralized environment.In a model of communication based on a random walk on an undirected graph, the optimal set over all sets of the same or smaller cardinality minimizes the sum of the mean first arrival times to the set by walkers starting at nodes outside the set...
متن کامل